1. Field of Invention
The present invention relates generally to machine vision, and more particularly, to a method of modeling the broadened-specular reflection of light to allow a machine to recognize the image of an object illuminated by varying lighting conditions.
2. Description of Related Art
It is clear that changes in lighting can have a large effect on how an object, such as a person, looks. Understanding this is essential to building effective object recognition systems.
U.S. patent application Ser. No. 09/705,507, filed 3 Nov. 2000, entitled “Lambertian Reflectance and Linear Subspaces”, now U.S. Pat. No. 6,853,745, the disclosure of which is hereby incorporated by reference, considered the relationship between the function that describes the lighting intensity, and the reflectance function that describes how much light an object reflects as a function of its surface normal, under a given lighting condition. Representing these functions as spherical harmonics, it was shown that for Lambertian reflectance the mapping from lighting to reflectance is a convolution with a nearly perfect low-pass filter. High-frequency components of the lighting hardly affect the reflectance function. Therefore, nearly all Lambertian reflectance functions could be modeled as some linear combination of nine spherical harmonic components.
These low-dimensional approximations have two key advantages. First, they are described with few parameters, thus optimization is simpler. Moreover, the images produced under low-frequency lighting are inherently immune to small changes in orientation. A small rotation in the surface normal is equivalent to a small rotation in the light. However, low-frequency light does not appreciably change with a small rotation. The analysis shows this, in that an image produced by an nth order harmonic light component is computed by taking an nth degree polynomial of the components of the surface normal.
While promising, these results considered only Lambertian reflectance. Specular reflectance can significantly affect how an object appears. For example, a human face may have some oiliness, particularly on the forehead, which reflects light specularly. This is seen as highlights on the object.
Specular reflectance cannot be approximated well by only considering low-dimensional linear subspaces which capture only low-frequency lighting. Intuitively, the set of possible reflectance functions for a perfect mirror is the same as the set of possible lighting functions, as will be shown, infra. The question remains how to cope with specularity in object recognition using models that will inevitably be imperfect.
Specularities are often thought of as mainly present in spatially localized highlights, which occur when a bright light source is reflected by a shiny object. Such highlights can potentially be identified in the image and processed separately. For example, the algorithms described in Coleman, et al., Obtaining 3-Dimensional Shape of Textured and Specular Surfaces Using Four-Source Photometry, Physics-Based Vision Principles and Practice, Shape Recovery, pp. 180–199 (1992), and Brelstaff, et al., Detecting Specular Reflection Using Lambertian Constraints, International Conference on Computer Vision, pp. 297–302 (1988), treat specular pixels as statistical outliers that should be discarded. However, specularities due to low-frequency lighting can be spread out throughout the entire image. Since these specularities affect every pixel in the image, their effect must be better understood; they cannot simply be discarded.
Some previous works have considered specular effects also as convolutions. Basri, et al., (August 2000) considered a version of the Phong model in which the specular lobe is a rotationally symmetric shape centered about a peak that is a 180-degree rotation of the direction of the incoming light about the surface normal. In such a model, specular reflection doubles the frequency of the incoming light, and then acts as a convolution. However, this model is heuristic, and not physically based. It does not extend to realistic models.
Ramamoorthi, et al., On the Relationship between Radiance and Irradiance: Determining the Illumination From Images of a Convex Lambertian Object, to appear in Journal of the Optical Society of America A, considered the effects of arbitrary bi-directional reflectance distribution functions (BRDFs) on the light field. They were able to develop algorithms for recovering the BRDF and/or the lighting given a measured light field and an object with a known 3D structure. Though interesting, these results are not generally applicable to machine vision because, without special sensing devices, the light field is not available. An image of an object is produced by the reflectance function that describes how the surface normals reflect light in one fixed direction. Therefore, an image corresponds to a 2D slice through the light field. For an arbitrary BRDF, the 2D reflectance function for a fixed viewpoint no longer results from a convolution of the lighting.